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ABSTRACT 

In  this  paper  we  deal  with  a  general  equality  constrained  minimi- 
zation problem.  The  concept  of  the  constrained  derivative  as  due  to 
Wilde  et  al. ,  [5]  -  [8]  is  extended  by  considering  more  general  parti- 
tions of  the  vector  of  unknowns  into  state  and  decision  components  and 
utilizing  the  properties  of  the  generalized  matrix  inverse.  The  main 
result  is  a  generalization  of  a  sufficient  condition  of  Wilde,  guaran- 
teeing the  existence  of  a  local  minimum. 


^Department  of  Business  Administration 
University  of  Illinois 
Urbana,  Illinois  61801 


GENERALIZED  CONSTRAINED  DERIVATIVES 

In  equality  constrained  rainimization  problems ,  a  test  for  a  local 
minimum  is  to  verify  the  stationarity  of  a  Lagrangian  and  the  definite- 
ness  of  a  certain  quadratic  form.  Using  the  concepts  of  state  and 
decision  variables  and  constrained  derivatives  Wilde  et  al.  [5]  -  [8] 
showed  that  stationarity  of  the  Lagrangian  and  definiteness  of  a  quadratic 
form  of  reduced  dimension  is  sufficient.   In  this  paper  we  introduce 
the  concept  of^a  generalized  constrained  derivative  and  derive  a 
generalization  of  Wilde's  sufficient  condition.  The  result  obtained  here 
does  not  depend  on  the  local  existence  of  a  non-singular  sub-Jacobian 
of  the  constraint  Jacobian,  but  rather  a  sxib-Jacobian  of  maximal  full 
celuBtn  rank. 

1.   NOTATION  AND  PRELIMINARIES 

Let  A  be  a  real  mxn  matrix.  We  denote  the  range  of  A  by  R(A)  and 
the  nulls pace  of  A  by  N(A).   The  transpose  of  A  is  denoted  by  A*.  The 
generalized  inverse  of  A  is  denoted  by  A"*".   If  A  is  square  and  non-singular 
then  A"*"  =  A~^,  the  inverse  of  A. 

For  references  on  the  generalized  inverse  the  reader  is  referred  to 
e.g.  [1]  and  [2].  A"*"  is  uniquely  defined  for  any  (raxn)  ve^l  matrix  A, 
and  computational  methods  for  determining  it  may  be;-found  for  example  in 
[23.   Some  particular  properties  of  A"*"  relevant  to  this  paper  are  now  given. 


(1.1)   AA"*"'  =  P 


D ( A 1 '  the  projection  onto  R(A), 


(1.2)   I  -  A'^'A  =  Pjj/.x»  the  projection  onto  N(A), 
Projection  maps  are  discussed  in  [4], 


In  view  of  (1.1)  and  (1.2)  we  have  the  following  result  on  linear 
equations:   If  yeR(A)  then  the  general  solution  to  Ax  =  y  is  given  by 

(1.3)   X  =  A'^'y  +  (I-A''"A)s   for  any  zeR". 

We  will  make  lise  of  the  following  analytic  notation.  Let  h:R"  ■*■  R^. 
Let  I [   I  I  be  the  Eucledian  norm.  Then 


h(t)  =  o|I  t  11^  means  lim  J^ =  0. 

*^  I!  t  !|k 

The  Jacob ian  of  h  at  t^,  if  it  exists,  will  be  denoted  (12.)   .  Here 


o 


and  the  Jacobian  is  the   (mxn)  matrix     |~j         r  jJHi — ^ I  inhere  (i  =  1,2,..., in). 


9h,-(to) 


f ah \       _  / 9hi(t 

Uj^^    \  3t, 


(j   =  1,2,...  ,n)   and  to  eR"^. 

If  m  =  1  we  write   [— J       =  V^h(tQ) ,  the  visual  gradient  notation. 


to 


Also,  when  m  =  1,  the  Hessian  of  h  at  t^,  if  it  exists,  i;:  -.h'-.   (nxn) 


/A 


matrix  r^^'^^oM  where  (i  =  1,2,. ..n)  and  (j  =  1,2, ...,n). 
I3ti3tjl 

2.   CONSTRAINED  DERIVATIVES 

In  this  section  a  brief  review  of  the  concepts  to  be  generalized  vril' 
be  presented.  The  problem  which  we  will  consider,  denoted  P,  is  given  by 

F-         minimize  y(x) 

f^(x)  =0  (i  =  1,. .. ,m) 

subject  to 

xeR 
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The  fHonctions  y,  f-,^,. . .  jf^^^  are  assumed  to  be  twice  differentiable  on 
r'^.  The  constraints  are  also  written  f(x)  =  0. 

Partition  the  vector  x  =  (-g-j  ,  where  ser'"  is  referred  to  as  the 
state  vector  and  deR  ~  is  called  the  decision  vector.  Assume  a  feasible 

"        ~  Lf\ "~ — 

point  Xq  is  such  that  the  Jacobian  [.^j   is  non-singular.  By  Taylor's 

l9slx^ 

theorem  '  ' 

(2.1)  3y  =  7^y(xQ)3d  +  Vgy(xo)9s  +  oI|3x|I 

If  Xq  is  a  feasible  point  then  by  Taylor's  theorem  the  feasibility  of 
x  +  9x  is  equivalent  to 

(2.2)  (lii        9s  +[lf|  3d  +  oM9x| 
9s /x^            »9d/x^ 


Since  f—\         is  invertible   (2.2)  yields 


''-''  ^^  =  -(C  te)x/^-°"^'^"" 


The  constrained  derivative  of  y  at  x  with  respect  to  the  partition 
i---|  is  defined  by 


(2 


,.)     (U)  ^^  =  ,,.(.„)  -  v.<s,  (If)  -  (II)  ^ 


From  (2.1)  -  (2.4)  we  obtain 
(2.5)    9y  =  [^  8d  +  o|!9xj| 

\^^}    Xq 

Since  9d  is  free  to  range  over  R^"""  we  have  the  following  result: 

A  necessary  condition  for  x,,  to  be  a  local  minimum  of  ¥  is  that  j^iLj   =  0. 

\oaJv 


Again  consider  problem  P  but  now  assume  y,  f,,  ...,  f^o  are  three  . 
times  differentiable.  Let  H  denote  the  Hessian  of  the  objective  function 
and  let  H^  denote  the  Hessian  of  the  ith  constraint  at  x  .  For  any  set 
of  numbers  (Xj^,  X2,  ...»  ^■^)   we  have 


(2.6)   9y  =  1- 

8x 


y-  E  X.f.    8x  +  i  ax*   IH  -  E  X.H.  j  3x  +  o| | 9x| | 

i=l     J       2  ^    i=i  -- 


If  (X  ,  ...  X  )  are  Lagrange  multipliers ,  that  is,  satisfy 


m 

m 


(2.7)  Vy(xQ),=  E  X.Vf^(x  ) 

i=l 

then 

/     m  _   \ 

(2.8)  9y  =  i  3x*  H  -   E   X^H^  |  Sx  +  o| 1 3x| j". 

Notice  that  a  unique  vector  of  Lagrange  multipliers  is  guaranteed  to 

exist  at  x  ,  by  our  assumption  on  j— ) 

m  _         \9s/Xq 

Write  P  =  H  -  E  X.  H..  Appropriate  partitioning  P,  we  have 
i=l  ■'■  """ 

(2.9)  3y  =  1     Od'^   9s*)        L!^^J~as_j  hd\     ^  o||9xl|^ 

\       CIS  I   SS  / 

Upon  defining 

and  making  use  of  (2.3)  and  (2.9)  we  obtain 

(2.10)  3y.=  J  3d*  S^^  +  o||3x||^ 

which  yields  the  following  result :  A  sufficient  condition  for  Xp  to  be  a 

m 
local  minimum  of  problem  F  is  that  the  Lagrangian  y  -  E  T.f .  is  stationary 

i=l  ^  ^ 
at  x^  and  S^^  is  positive  definite. 


All  the  material  in  section  2  can  be  found  in  [7], 

3.   GENERALIZED  CONSTRAINED  DERIVATIVES 

Again  consider  problem  P,  with  the  functions  y,  f-|_,  ...,  f^^  being 
twice  differentiable  where  x  is  a  feasible  point.  Now  take  a  partition 
y  =  (-J-)  where  s  is  not  necessarily  an  Tn-vector. 

An  auxiliary  problem,  denoted  P   ,  is  noftt   introduced: 


^O 


P^      ■*  minimize  y(x) 

s^iectto   ^M.)    my      f(x)  =  0 
^  'x^   ^  'x„ 


n 
xeR 


Assume  thatf — -j   is  a  maximal  full  column  rank  submatrix  of  {—]     . 
Feasibility  of  x^  +  5^^  for  problem  P^^  is  equivalent  to 

Since  l^\  |li.\         -  P      /  ,     .         \  and  by  the  above  assumption  on  |— |        , 

l3sL  l3sL  R       3i\  ^  Ui 


"o 
(3.1)  may  be  rewritten  as  follows: 


o      i\a<=/    I  o 


"•^>  (fi  '^*(i)  ''i^]  (sr°ii-ii=° 


'^O  '^o 
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Since  all  three  terms  in  this  equation  are  in  R  i — ^1   we  can  solve  for  3s 

and  obtain 

^o     '^o 

The  generalized  constrained  derivative  of  y  at  x^  with  respecf  to 
the  partition  |-|-|  is  defined  by 

(3..,  (|.)  -   v,y(x„,  -  V(.<o'  (f);  (||}_^ 

'^o  -.   '  '^o    ^o 


Similarly  to  (2.5)  we  have 
(3.5)   ay  =  f^       9d  +  oi  |3x|  | 


^o 


and  the  following  fact: 

A  necessary  condition  for  x^^  to  be  a  local  minimum  of  P   is  that 


^o 


^\   =  0. 

It  is  not  necessarily  true  that  the  generalized  constrained  derivative  =  0 


V^d.Xo 


at  a  local  minimum  of  problem  P,  however,  as  is  evidenced  by  the  following 

counterexample.   Consider  the  problem 

...    ,  .     2     2 
minimize  y(x)  =  x   +  x   +  x 

9  1 

subject  to         f-j^(x)  =  3<j  -  ^2     ~  ^3     ~  ^ 

f^Cx)   =  Xg^   ^  0 

The  solution  obviously  is  (0,0,0)  and  (— -j   =  (     j.  Upon  setting 

/x  \  IT  '•*  o   ^ 

s  =  X,  ,  d  ~  f  2|we  obtain  f=Xj  =  (0,1). 
^-      1^3/  Ud/ 


3' 
This  occurs  for  the  following  reason:   In  expression  (2.2),  when 


s  not  square  ,  the  terms  o  j | 3x | |  need  not  be  in  R  { —  j   ,  and  one 


o 


-6- 


cannot  proceed  to  solve  (2.2)  for  3s  by  using  a  generalized  inverse. 
This,  it  can  be  verified,  is  precisely  the  case  in  the  aboye  counter- 
example. This  difficulty  cannot  occur,  of  course,  if  the  constraints 
in  problem  P  are  linear;  that  is,  f(x)  =  Ax-b.  The  generalized  constrained 
derivative  for  a  problem  with  linear  constraints  is 


•  ^l^j  =  V^y(xo)  -  V^yCx^)  A^  A^ 

where   A  =  (A^  '  A,)  =  f™V 
•^  i  a    \3x/ 


For  the  case  where  P  has  linear  constraints  it  is  necessary  •''hat  the 
generalized  constrained  derivative  be  0  at  a  local  minimum.   If  y(x)  is 
linear  and  Ag  is  non-singular,  then  the  last  statement  is  equivalent  to 
saying  that  the  simplex  criterion  usually  denoted  by  z-j-c  is  0  for  variables 
X.  in  the  basis.  Different  partitions  I ---j correspond  to  different  vertices 
of  the  feasible  region.  The  state  variables  are  the  basic  variables  and 
the  decision  variables  are  the  variables  not  in  the  basis.   (See  e.g.  [3]). 
Observe  that 


(3.6)   X  is  a  local  minimum  solution  of  P   ==£>  x  is  a  local  minimum 

O  Xq         o 

solution  of  P 
Thus  a  sufficient  condition  for  x,^  to  be  a  local  miniraini  of  P   is 

O  V 

^O 

sufficient  condition  for  P  as  well.  Such  a  condition  will  now  be  derived. 

Let  X  be  a  feasible  point  for  problem  P.   It  is  clearly  then  feasible 

for  P„  .   Again  assume  that  ( — |   is  a  maximal  full  column  rank  submatrix  of 
^o  V^sL 

/8f\  ° 

l~~-]     .     AI90,  assume  now  that  the  functions  y,   f,  ,    ...,  fj„  are  three  times 


^'O 


differentiable.  Assume 


(3,7)    Vy(x  )  G  R   fM^ 

o 

This  implies  the  existence  of  a  (not  necessarily  unique)  vector  of  Lagrange 
multipliers  ( A-,  ,  . . .  ,  A  ) . ' 

Following  the  development  from  (2.6)  -  (2.10),  using  (3.3)  in  place 
of  (2.3),  and  (3,6),  we  see  that  a  sufficient  condition  for  x„  to  he  a  local 

■  ■  I  >  - 1  ■  .   o      III 

rainimuiTi  of  P  is  that  (3.7)  holds  and  S*   is  positive  definite.  Here  Si, 
is  the  square  matrix 

^dd  ~  ^dd  "  ^ds 


(i):  (M) 


\ft 


p     fMV     /9i\ 

ss    \3sl  \^^] 

Xq/  x^  x^ 


m 


where  P  =  H  -  Z  A.H.  is  appropriately  partitioned. 
1=1 


CONCLUDING  REMARKS 


/9f\ 
Upon  locating  a  non-singular  sub-Jacobian  of  (-— 1   and  verifying 

_  ■^o 

that  the  Lagrangian  of  problem  P  is  stationary  at  x  for  a  Lagrange 

multiplier  A,  Wilde  has  defined  a  matrix  of  "constrained  second  derivatives," 

B^,...  vrhose  definition  involves  a.   S,,  is  of  dimension  (n-m  x  n-m) , 

where  rn  is  the  nuirher  of  constraints.   The  definiteness  of  Sj,  guarantees 

that  X  is  a  local  sxtremum  of  problem  P. 

In  this  paper  we  have  treated  the  case  where  a  non-singular  sub- 

Jacobijan  may  not  exist ,  but  we  considered  maximal  full  column-rank  sub- 

Jacobians.  •  By  5-ntroducing  an  "artificial"  problem,  P^  ,  and  using  the 

Xq 

properties  of  the  genereilized  matrix  inverse,  vre  defined  SI,,  which  is  an 
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(n-p  X  n-p)  analog  of  S^^.  Here  p  <_  m.  Under  stationarity  of  the  Lagrangian 
of  F  the  definiteness  of  S^^  guarantees  a  local  extremura.   It  can  be  seen, 
by  following  the  arguments  of  section  3,  that  if  p  =  m,  then  our  result  is 
precisely  that  of  Wilde. 
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